186 research outputs found
Probabilistic movement primitives
Movement Primitives (MP) are a well-established approach for representing modular
and re-usable robot movement generators. Many state-of-the-art robot learning
successes are based MPs, due to their compact representation of the inherently
continuous and high dimensional robot movements. A major goal in robot learning
is to combine multiple MPs as building blocks in a modular control architecture
to solve complex tasks. To this effect, a MP representation has to allow for
blending between motions, adapting to altered task variables, and co-activating
multiple MPs in parallel. We present a probabilistic formulation of the MP concept
that maintains a distribution over trajectories. Our probabilistic approach
allows for the derivation of new operations which are essential for implementing
all aforementioned properties in one framework. In order to use such a trajectory
distribution for robot movement control, we analytically derive a stochastic feedback
controller which reproduces the given trajectory distribution. We evaluate
and compare our approach to existing methods on several simulated as well as
real robot scenarios
Model-free Probabilistic Movement Primitives for physical interaction
Physical interaction in robotics is a complex problem
that requires not only accurate reproduction of the kinematic
trajectories but also of the forces and torques exhibited
during the movement. We base our approach on Movement
Primitives (MP), as MPs provide a framework for modelling
complex movements and introduce useful operations on the
movements, such as generalization to novel situations, time
scaling, and others. Usually, MPs are trained with imitation
learning, where an expert demonstrates the trajectories. However,
MPs used in physical interaction either require additional
learning approaches, e.g., reinforcement learning, or are based
on handcrafted solutions. Our goal is to learn and generate
movements for physical interaction that are learned with imitation
learning, from a small set of demonstrated trajectories.
The Probabilistic Movement Primitives (ProMPs) framework
is a recent MP approach that introduces beneficial properties,
such as combination and blending of MPs, and represents the
correlations present in the movement. The ProMPs provides
a variable stiffness controller that reproduces the movement
but it requires a dynamics model of the system. Learning such
a model is not a trivial task, and, therefore, we introduce the
model-free ProMPs, that are learning jointly the movement and
the necessary actions from a few demonstrations. We derive
a variable stiffness controller analytically. We further extent
the ProMPs to include force and torque signals, necessary for
physical interaction. We evaluate our approach in simulated
and real robot tasks
Sample-based information-theoretic stochastic optimal control
Many Stochastic Optimal Control (SOC) approaches
rely on samples to either obtain an estimate of the
value function or a linearisation of the underlying system model.
However, these approaches typically neglect the fact that the
accuracy of the policy update depends on the closeness of the
resulting trajectory distribution to these samples. The greedy
operator does not consider such closeness constraint to the
samples. Hence, the greedy operator can lead to oscillations
or even instabilities in the policy updates. Such undesired
behaviour is likely to result in an inferior performance of the
estimated policy. We reuse inspiration from the reinforcement
learning community and relax the greedy operator used in SOC
with an information theoretic bound that limits the ‘distance’ of
two subsequent trajectory distributions in a policy update. The
introduced bound ensures a smooth and stable policy update.
Our method is also well suited for model-based reinforcement
learning, where we estimate the system dynamics model from
data. As this model is likely to be inaccurate, it might be
dangerous to exploit the model greedily. Instead, our bound
ensures that we generate new data in the vicinity of the current
data, such that we can improve our estimate of the system
dynamics model. We show that our approach outperforms
several state of the art approaches on challenging simulated
robot control tasks
Extracting low-dimensional control variables for movement primitives
Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset
A probabilistic approach to robot trajectory generation
Motor Primitives (MPs) are a promising approach
for the data-driven acquisition as well as for the modular and
re-usable generation of movements. However, a modular control
architecture with MPs is only effective if the MPs support
co-activation as well as continuously blending the activation
from one MP to the next. In addition, we need efficient
mechanisms to adapt a MP to the current situation. Common
approaches to movement primitives lack such capabilities or
their implementation is based on heuristics. We present a
probabilistic movement primitive approach that overcomes the
limitations of existing approaches. We encode a primitive as a
probability distribution over trajectories. The representation as
distribution has several beneficial properties. It allows encoding
a time-varying variance profile. Most importantly, it allows
performing new operations — a product of distributions for
the co-activation of MPs conditioning for generalizing the MP
to different desired targets. We derive a feedback controller
that reproduces a given trajectory distribution in closed form.
We compare our approach to the existing state-of-the art and
present real robot results for learning from demonstration
Environmental influences on affect and cognition: A study of natural and commercial semi-public spaces
Research has consistently shown differences in affect and cognition after exposure to different physical environments. The time course of these differences emerging or fading during exploration of environments is less explored, as most studies measure dependent variables only before and after environmental exposure. In this within-subject study, we used repeated surveys to measure differences in thought content and affect throughout a 1-h environmental exploration of a nature conservatory and a large indoor mall. At each survey, participants reported on aspects of their most recent thoughts (e.g., thinking of the present moment vs. the future; thinking positively vs. negatively) and state affect. Using Bayesian multi-level models, we found that while visiting the conservatory, participants were more likely to report thoughts about the past, more positive and exciting thoughts, and higher feelings of positive affect and creativity. In the mall, participants were more likely to report thoughts about the future and higher feelings of impulsivity. Many of these differences in environments were present throughout the 1-h walk, however some differences were only evident at intermediary time points, indicating the importance of collecting data during exploration, as opposed to only before and after environmental exposures. We also measured cognitive performance with a dual n-back task. Results on 2-back trials replicated results from prior work that interacting with nature leads to improvements in working-memory performance. This study furthers our understanding of how thoughts and feelings are influenced by the surrounding physical environment and has implications for the design and use of public spaces
- …